Skip to content

CB: added PerfMetrics support #1524

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 14 commits into from
Jan 14, 2025
Merged

CB: added PerfMetrics support #1524

merged 14 commits into from
Jan 14, 2025

Conversation

pavel-esir
Copy link
Contributor

@pavel-esir pavel-esir commented Jan 10, 2025 •
edited
Loading

image

Ticked: CVS-158715

@pavel-esir pavel-esir added bug Something isn't working enhancement New feature or request labels Jan 10, 2025
@github-actions github-actions bot added category: continuous batching Continuous batching category: speculative decoding Speculative decoding category: CPP API Changes in GenAI C++ public headers no-match-files category: prompt lookup Prompt look-up decoding labels Jan 10, 2025
@ilya-lavrenov ilya-lavrenov changed the title Add ContinuousBatching PerfMetrics CB: added PerfMetrics support Jan 11, 2025
@ilya-lavrenov ilya-lavrenov added this to the 2025.0 milestone Jan 11, 2025
@ilya-lavrenov ilya-lavrenov self-assigned this Jan 11, 2025
ilya-lavrenov
ilya-lavrenov suggested changes Jan 13, 2025
View reviewed changes
@github-actions github-actions bot removed category: speculative decoding Speculative decoding category: prompt lookup Prompt look-up decoding labels Jan 13, 2025
@pavel-esir pavel-esir marked this pull request as ready for review January 13, 2025 10:59
ilya-lavrenov
ilya-lavrenov reviewed Jan 13, 2025
View reviewed changes
@github-actions github-actions bot added the category: LLM LLM pipeline (stateful, static) label Jan 13, 2025
pavel-esir
pavel-esir commented Jan 13, 2025
View reviewed changes
pavel-esir
pavel-esir commented Jan 13, 2025
View reviewed changes
pavel-esir and others added 2 commits January 13, 2025 13:49
@pavel-esir
corrections
3b33e7d
@pavel-esir @ilya-lavrenov
Apply suggestions from code review
faef43d 
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
pavel-esir
pavel-esir commented Jan 13, 2025
View reviewed changes
ilya-lavrenov
ilya-lavrenov reviewed Jan 13, 2025
View reviewed changes
@github-actions github-actions bot added the category: Python API Python API for GenAI label Jan 13, 2025
ilya-lavrenov
ilya-lavrenov reviewed Jan 13, 2025
View reviewed changes
Copy link
Contributor

@ilya-lavrenov ilya-lavrenov left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, re-generate .pyi file for Python API

pavel-esir and others added 3 commits January 13, 2025 14:25
@pavel-esir @ilya-lavrenov
Apply suggestions from code review
44eb13f 
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
@pavel-esir
add PyAPI
b159e64
@pavel-esir @ilya-lavrenov
Update src/cpp/src/icontinuous_batching.cpp
1f6540a 
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
@pavel-esir
Copy link
Contributor Author

Please, re-generate .pyi file for Python API

done

@ilya-lavrenov ilya-lavrenov removed the bug Something isn't working label Jan 13, 2025
@ilya-lavrenov ilya-lavrenov added this pull request to the merge queue Jan 13, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 13, 2025
@ilya-lavrenov ilya-lavrenov merged commit a322f01 into openvinotoolkit:master Jan 14, 2025
59 checks passed
@pavel-esir pavel-esir deleted the add_cp_perf_metrics branch January 14, 2025 06:18
@ilya-lavrenov ilya-lavrenov mentioned this pull request Jan 14, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ilya-lavrenov ilya-lavrenov ilya-lavrenov approved these changes

Assignees

@ilya-lavrenov ilya-lavrenov

Labels
category: continuous batching Continuous batching category: CPP API Changes in GenAI C++ public headers category: LLM LLM pipeline (stateful, static) category: Python API Python API for GenAI enhancement New feature or request no-match-files
Projects
None yet
Milestone
2025.0
Development

Successfully merging this pull request may close these issues.
2 participants
@pavel-esir @ilya-lavrenov